Enrollment Using Synthetic Fingerprint Image and Fingerprint Sensing Systems

ABSTRACT

A fingerprint sensing system. The fingerprint sensing system includes: at least one sensor; at least one display device; at least one application processor; and at least one secure enclave processor. The application processor(s) receives fingerprint data from the sensor(s) and provides the fingerprint data to the secure enclave processor(s). The secure enclave processor(s) decodes the fingerprint data and provides a signal indicative of at least one matched node. The application processor(s), responsive to receipt of the signal indicative of the matched node(s), presents at least a portion of a synthetic fingerprint image via at least one display device corresponding to the matched node(s).

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 17/141,088, filed Jan. 4, 2021, which is a continuation of U.S.patent application Ser. No. 16/287,943, filed Feb. 27, 2019, now U.S.Pat. No. 10,885,293, which is a continuation of U.S. patent applicationSer. No. 15/607,354, filed May 26, 2017, now U.S. Pat. No. 10,255,474,which is a continuation of U.S. patent application Ser. No. 14/566,495,filed Dec. 10, 2014, now U.S. Pat. No. 9,665,785, which is acontinuation of U.S. patent application Ser. No. 14/243,858, filed Apr.2, 2014, now U.S. Pat. No. 8,913,802, which is a continuation of U.S.patent application Ser. No. 13/802,695, filed Mar. 13, 2013, now U.S.Pat. No. 8,913,801, which claims the benefit under 35 U.S.C. § 119(e) toU.S. Provisional Patent Application No. 61/666,595, which was filed onJun. 29, 2012, all of which are incorporated by reference as if fullydisclosed herein.

TECHNICAL FIELD

Embodiments described herein relate generally to fingerprint sensingsystems and examples of enrollment using synthetic fingerprint imagesare described.

BACKGROUND DESCRIPTION

Fingerprint sensing technology has become widespread in use and is oftenused to provide secure access to sensitive electronic devices and/ordata. Generally, capacitive fingerprint sensors may be used to determinean image of a fingerprint through measuring capacitance through multiplecapacitive sensing elements. The higher the capacitance, the nearer thesurface of an adjacent or overlying finger to the capacitive sensingelement. Thus, fingerprint ridges provide a higher capacitance to anunderlying capacitive sensing element than do fingerprint valleys.

Data generated by a fingerprint sensor may be encrypted for securitypurposes and processed using a secure processor. A fingerprint match toa known fingerprint image may also be determined by the secureprocessor. In this manner, fingerprint data may be kept secure and maynot be exposed unencrypted or unsecured to non-secure processors.

Fingerprint images may generally be considered to be made up of several‘nodes’, with each node representing a region of the fingerprint image.Nodes may generally be overlapping, such that the nodes may be stitchedtogether to form an entire fingerprint image.

Fingerprint sensors may be smaller than the fingerprint, and a user maybe required to roll, swipe, or otherwise move their finger to exposedifferent regions of the finger to the fingerprint sensor in order forthe system to obtain a complete fingerprint image.

SUMMARY

One sample embodiment, as described herein, is a fingerprint sensingsystem. The fingerprint sensing system includes: at least one sensor; atleast one display device; at least one application processor; and atleast one secure enclave processor. The application processor(s)receives fingerprint data from the sensor(s) and provides thefingerprint data to the secure enclave processor(s). The secure enclaveprocessor(s) decodes the fingerprint data and provides a signalindicative of at least one matched node. The application processor(s),responsive to receipt of the signal indicative of the matched node(s),presents at least a portion of a synthetic fingerprint image via atleast one display device corresponding to the matched node(s).

Another example embodiment, as described herein, is a method offingerprint sensing, the method includes receiving fingerprint data fromat least one sensor utilizing at least one application processor. Thefingerprint data is provided to at least one secure enclave processorutilizing the application processor(s). The secure enclave processor(s)decodes the fingerprint data and provides a signal indicative of atleast one matched node. Responsive to receipt of the signal indicativeof the matched node(s), at least a portion of a synthetic fingerprintimage is presented via at least one display device, the portion of thesynthetic fingerprint image corresponding to the matched node(s)utilizing the application processor(s).

A further example embodiment, as described herein, is a fingerprintsensing system. The fingerprint sensing system includes: a sensor; adisplay device; an application processor; and a secure enclaveprocessor. The application processor is configured to provide at least aportion of a synthetic fingerprint image for display on the displaydevice. The application processor is also configured to receivefingerprint data from the sensor and provide the fingerprint data to thesecure enclave processor. The secure enclave processor is configured todecode the fingerprint data and provide a signal indicative of a matchednode. The application processor, responsive to receipt of the signalindicative of the matched node, is configured to change a portion of thesynthetic fingerprint image corresponding to the matched node.

While multiple embodiments are disclosed, including variations thereof,still other embodiments of the present disclosure will become apparentto those skilled in the art from the following detailed description,which shows and describes illustrative embodiments of the disclosure. Aswill be realized, the disclosure is capable of modifications in variousobvious aspects, all without departing from the spirit and scope of thepresent disclosure. Accordingly, the drawings and detailed descriptionare to be regarded as illustrative in nature and not restrictive.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic illustration of a fingerprint sensing system inaccordance with an embodiment of the present invention.

FIG. 2 is a schematic illustration of a synthetic fingerprint imagedisplayed on a display device in accordance with an embodiment of thepresent invention.

FIG. 3 is a schematic illustration of a synthetic fingerprint imagedisplayed on a display device during an initial portion of an enrollmentprocess in accordance with an embodiment of the present invention.

FIG. 4A is a schematic illustration of a synthetic fingerprint imagedisplayed on a display device during an enrollment process in accordancewith an embodiment of the present invention.

FIG. 4B is a flow chart illustrating a method for using the fingerprintsensing system to display feedback to a user.

FIG. 5 is a schematic illustration of a system in accordance with anembodiment of the present invention.

FIG. 6 is a block diagram of one embodiment of a system arranged inaccordance with an embodiment of the present invention.

FIG. 7 is a flowchart illustrating an example method of fingerprintsensing, as described herein.

DETAILED DESCRIPTION

The use of fingerprint sensing technology to provide secure access tosensitive electronic devices and/or data is gaining in popularity.Embodiments described herein may be configured to operate with a varietyof sensors, including strip or swipe sensors, array or othertwo-dimensional sensors, and the like. Capacitive fingerprint sensorsare one technology that may be used to determine the image of afingerprint.

Because fingerprint sensors may be smaller than the fingerprint, a usermay be required to roll, swipe, or otherwise move their finger to exposedifferent regions of the finger to the fingerprint sensor for the systemto obtain a complete fingerprint image. However, it may be difficult forthe user to be certain a sufficient amount of their finger has beenexposed to the fingerprint sensor. Further, the user may not be awarethat certain portions of their finger were not exposed to the sensor(e.g. missed the sensor or were never intended to be placed on thesensor). Accordingly, it may be useful for the fingerprint sensor systemto provide a feedback mechanism to inform a user which portions of theirfinger have passed over the sensor (or, in some examples, which portionsof their finger have been matched to stored data).

Thus, example embodiments described herein, may display an image of afingerprint on a display device during an enrollment process or testingprocess to provide feedback to a user regarding portions of the fingerwhich need to be presented to the sensor. However, it may be undesirableto display an image of an actual user's fingerprint on the display forsecurity reasons. Accordingly, it may be useful to display a syntheticfingerprint rather than the user's actual fingerprint for theseoperations.

The following terminology is exemplary, and not intended to be limitingin any way.

The text “capacitive sensing element”, and variants thereof, generallyrefers to one or more sensors that may sense data elements of any kind,including information sensed with respect to individual locations. Forexample and without limitation, a capacitive sensing element may sensedata or other information with respect to a relatively small region of afingerprint image.

After reading this application, those skilled in the art would recognizethat these statements of terminology would be applicable to techniques,methods, physical elements, and systems (whether currently known orotherwise), including extensions thereof inferred or inferable by thoseskilled in the art after reading this application.

Generally, examples described herein include fingerprint sensing systemsthat may facilitate enrollment using a synthetic fingerprint image.Enrollment includes a process by which a user may roll, swipe, orotherwise move their finger across a fingerprint sensor so thefingerprint sensing system may obtain fingerprint data corresponding tothe user's finger. The fingerprint sensing system during enrollment maymatch the fingerprint data corresponding to the user's finger to known,stored, fingerprint data to identify (e.g. authenticate) the user.

It may be difficult for the user to be certain a sufficient amount oftheir finger has been exposed to the fingerprint sensor. For example,the user may not be aware that certain portions of their finger were notexposed to the sensor (e.g. missed the sensor or were never intended tobe placed on the sensor). Accordingly, examples of the present inventionmay provide a feedback mechanism to inform a user which portions oftheir finger have passed over the sensor (or, in some examples, whichportions of their finger have been matched to stored data).

In some examples, an image of a fingerprint is displayed on a displayscreen viewable by the user to inform the user which portions of theirfinger no longer need to be passed over the sensor to completeenrollment. As will be described further below, portions of thedisplayed fingerprint may be filled in, or otherwise changed, reflectingthe portions which are no longer needed. However, it may not bedesirable to display an image of the actual fingerprint data beingcollected by the fingerprint sensor, because doing so may compromise thesecurity of the fingerprint data (e.g. if malicious screen capturesoftware is running on the device that may capture the displayedfingerprint data). Accordingly, examples described below may provide fora synthetic fingerprint image to be displayed and to provide feedback tothe user by altering portions of the synthetic fingerprint image duringan enrollment process. The synthetic fingerprint image may be a staticpredetermined image or in some examples the synthetic fingerprint imagemay be generated using the actual received fingerprint information fromthe fingerprint sensor, or may be generated using other factors—e.g.time of day.

FIG. 1 is a schematic illustration of a fingerprint sensing system 100according to an embodiment of the present invention. The fingerprintsensing system 100 includes a sensor 105, a processing platform 110 anda display device 112. Other components, including user interfaceelements, may be included in the fingerprint sensing system 100 in otherembodiments.

The sensor 105 may include any type of fingerprint sensor, such as acapacitive sensor or capacitive sensing element that may generatesignals corresponding to ridges and valleys of a fingerprint. The sensor105 may include an active area 115 that is responsive to ridges andvalleys placed on the active area 115. The sensor 105 may be below oradjacent a user interface element, such as a button 117. The active area115 of the sensor 105 may be smaller than an entire fingerprint in someembodiments. Accordingly, as has been described above, a user may berequired to move a finger around in order to expose the entirefingerprint to the active area 115.

The processing platform 110 may include any number of processingunit(s), each of which may be implemented by a processor, and computerreadable storage media sufficient to store executable instructions that,when executed by the processing unit(s), may cause the processingunit(s) to perform the functions described herein. The actual hardwareand software configuration of the processing platform 110 may vary indifferent embodiments. The processing platform 110 includes anapplication processor (AP) 120 and a secure enclave processor (SEP) 122.The application processor 120 may be implemented using one or moresuitable processors, and the secure enclave processor 122 may also beimplemented using one or more processors, which processors are differentthan those used to implement the application processor 120. In someexamples, the application processor 120 may be implemented using adevice's central processing unit (CPU), while the secure enclaveprocessor 122 may be a separate processor.

The secure enclave processor 122 is generally used to manipulate securedata (e.g. to decrypt encrypted fingerprint data from the sensor 105 andmatch the decrypted data with stored fingerprint template data in atemplate library 124). The template library 124 may be stored on anycomputer readable storage medium (e.g. memory) in communication with thesecure enclave processor 122, and one or more distinct computer readablestorage media may be used to implement the template library 124. Thesecure enclave processor 122 may have access to a key or other securityparameter usable to decrypt data received from the sensor 105. Forexample, the secure enclave processor 122 and the sensor 105 may share afactory provisioned key, enabling the secure enclave processor 122 todecrypt data received from the sensor 105. The application processor 120may not have access to the key or other security parameter, and may beunable to decrypt data received from the sensor 105. In this manner, theapplication processor 120 may be prevented from ever accessing decrypteddata from the fingerprint sensor 105, which may improve the security ofthe fingerprint data, for example, making the decrypted fingerprint datainaccessible or less accessible to other programs which may be runningon the application processor 120.

Embodiments of the present invention may display an image of afingerprint on a display device, such as the display device 112 duringan enrollment process to provide feedback to a user regarding portionsof the finger which need to be presented to the sensor 105. However, itmay be undesirable to display an image of an actual user's fingerprinton the display for security reasons. Accordingly, the applicationprocessor 120 may include a synthetic fingerprint generator 130. Thesynthetic fingerprint generator 130 may be implemented, for example by acomputer readable storage medium or a plurality of such computerreadable storage media (e.g. one or more memories) storing executableinstructions for generating a synthetic fingerprint. The applicationprocessor 120 may execute the instructions for generating a syntheticfingerprint and cause the synthetic fingerprint to be displayed on thedisplay device 112.

The synthetic fingerprint generator 130 may generate the syntheticfingerprint image in a variety of ways. The synthetic fingerprint isgenerally an image resembling a fingerprint but which is not a replicaof the user's actual fingerprint. In some examples, a syntheticfingerprint image may be selected from a library of stored syntheticfingerprint images, which may be stored on a computer readable storagemedium (e.g. memory) accessible to the application processor 120. Insome examples, the synthetic fingerprint generator 130 may generate orselect a synthetic fingerprint image whose features are determined basedon some aspect of the enrollment process, including, but not limited to,time of day, type of device, device ID, node of fingerprint firstcaptured, or combinations thereof.

During operation, a user may place all or a portion of their finger overthe sensor 105. The sensor 105 may provide encrypted fingerprint data tothe processing platform 110. The encrypted fingerprint data may bereceived by the application processor 120, however, the applicationprocessor 120 may not be able to decrypt the fingerprint data. Theapplication processor 120 may provide the encrypted fingerprint data tothe secure enclave processor 122. Moreover, the application processor120 may, responsive to the initiation of an enrollment process, providea synthetic fingerprint image on the display device 112. As describedabove, the synthetic fingerprint image may be provided from a storedimage or may be generated responsive to an aspect of the enrollmentprocess.

The secure enclave processor 122 may accordingly receive the encryptedfingerprint data, decrypt the fingerprint data, and search for a matchin the template library 124. Generally, fingerprint data of nodes of afinger may be matched to stored fingerprint data in the template library124. For example, the sensor 105 may not provide data at oncerepresenting an entire fingerprint image, but may provide fingerprintdata representing a node (e.g. a portion of a fingerprint). The secureenclave processor 122 may match the node with data stored in thetemplate library 124 to find a match. The secure enclave processor 122may provide a signal indicative of a match, or lack thereof, to theapplication processor 120. The secure enclave processor 122 may furtherprovide an indication to the application processor 120 of which node ofthe fingerprint was matched.

The application processor may display all or a portion of the syntheticfingerprint image on the display device 112. As nodes of the fingerprintare received and matched by the secure enclave processor, theapplication processor may cause corresponding portions of the syntheticfingerprint image displayed on the display device 112 to change, therebyproviding feedback to the user of which portions of the fingerprint hadbeen successfully received, and in some examples matched, by the secureenclave processor 122. The application processor 120 may provide thisfeedback by executing stored executable instructions for enrollmentfeedback, which may be stored in a computer readable storage medium(e.g. memory) accessible to the application processor 120.

FIG. 2 is a schematic illustration of a synthetic fingerprint imagedisplayed on a display device in accordance with an embodiment of thepresent invention. The display device 200 may be implemented using thedisplay device 112 of FIG. 1 . The synthetic fingerprint image 205 maybe a schematic representation of a fingerprint which may be displayedduring enrollment, but which may not be an accurate representation ofthe fingerprint being enrolled, as has been described above. Thedifferent nodes of a fingerprint are shown schematically with dottedlines in FIG. 2 . The grid 210 depicts regions of 16 nodes that mayrepresent a fingerprint, for example, the node 215 is near a center ofthe fingerprint. The nodes are shown adjacent one another in FIG. 2 forease of illustration, but in some embodiments of the present inventionthe nodes may overlap, allowing the nodes to be stitched together toform a completed fingerprint image. The grid 210 or other delineation ofnodes may or may not itself be displayed on the display device 200.

FIG. 3 is a schematic illustration of a synthetic fingerprint imagedisplayed on a display device during an initial portion of an enrollmentprocess in accordance with an embodiment of the present invention. Inembodiments of the present invention, an indicator may be displayed onthe display device 200 illustrating which node of a finger the usershould attempt to present first, or next, to a sensor. The displaydevice 200 of FIG. 3 may be implemented, for example, by the displaydevice 112 of FIG. 1 . The application processor as described in FIG. 1may display the indication in accordance with computer executableinstructions for the same stored in a medium accessible to theapplication processor. The indicator 305 may be a box, as illustrated inFIG. 3 , and may be shaded, colored, outlined, or otherwise differentthan the surrounding image. In some examples, the indicator 305 may nothave a contour, but portions of the synthetic fingerprint image 205corresponding to the node to be indicated may instead be displayed in adifferent color, dashing, thickness, or otherwise different than theremaining synthetic fingerprint image 205.

FIG. 4A is a schematic illustration of a synthetic fingerprint imagedisplayed on a display device during an enrollment process in accordancewith an embodiment of the present invention. In embodiments of thepresent invention, portions of a synthetic fingerprint image displayedon the display device 200 may change during the enrollment process toindicate to a user that those portions have been captured (e.g. matched)by the device. The display device 200 of FIG. 4A may be implemented, forexample, by the display device 112 of FIG. 1 . The application processoras described in FIG. 1 may change portions of the synthetic fingerprintimage in accordance with computer executable instructions for the samestored in a medium accessible to the application processor. Portions ofthe synthetic fingerprint image may change, for example, by changing adash pattern, color, shading, thickness, etc. of the lines making up thesynthetic fingerprint image and/or regions of the synthetic fingerprintimage, including the background. In the example of FIG. 4A, thesynthetic fingerprint image 205 is shown in dashed lines. As a userenrolls a finger, portions (e.g. nodes) of the finger which aresuccessfully captured may be displayed in solid lines, e.g. the node 405shown in FIG. 4A has been successfully captured and is shown in solidlines. By changing portions of the synthetic fingerprint image duringenrollment, a user may receive feedback as to which portions of thefinger still need to be presented to a sensor, such as the sensor 105 ofFIG. 1 .

During operation, referring back to FIG. 1 , the application processor120 may cause the display device 112 to display a synthetic fingerprintimage. The application processor 120 may further cause the displaydevice 112 to display an indication of an initial node for fingerprintenrollment, in accordance with stored executable instructions. Duringenrollment, fingerprint data may be received by the applicationprocessor 120 from the sensor 105 and passed to the secure enclaveprocessor 122. The secure enclave processor 122 may decode thefingerprint data and match the fingerprint data with data stored in thetemplate library 124. The secure enclave processor 122 may provide asignal to the application processor 120 indicative of a matched node ofthe fingerprint. The application processor may accordingly cause thesynthetic fingerprint image displayed on the display device 112 tochange to indicate matching of the particular node has occurred.

An illustrative method for using the fingerprint sensing system will nowbe discussed in further detail. FIG. 4B is a flow chart illustrating amethod for sensing fingerprint data and providing feedback to a user.The method 400 may begin with operation 402 and the sensor 105 maycapture data corresponding to one or more fingerprints of a user. Inother words, the sensor 105 may capture data corresponding to one ormore features (examples of which include valleys and ridges) on a user'sfinger.

Once the initial fingerprint data has been captured, the method 400 mayproceed to operation 404. In operation 404, the display device 112 maydisplay feedback to the user. The feedback may include the syntheticfingerprint image, text, or the like. The feedback may correspond to thedata received in operation 402. For example, the feedback may indicateto a user the portions of finger that were captured by the sensor 105,may provide a percentage of the finger surface sensed by the sensor 105,or the like.

After operation 404, the method 400 may proceed to operation 406. Inoperation 406, the processor 122 may determine whether additional datais required. For example, the processor may determine whether asufficient number of nodes were captured by the sensor 105 in order toevaluate the fingerprint data to determine if there is a match. Thenumber of nodes may depend on the size of the finger, the accuracy orsensitivity of the initial captured data, desired match accuracy, and soon.

In operation 406, if additional data is required, the method 400 mayproceed to operation 408. In operation 408, the sensor 105 may beconfigured to capture additional fingerprint data. As one example, thedisplay device 112 may provide output to a user requesting that he orshe provide additional input to the sensor 105. In another example, thesensor 105 may automatically scan the sensing surface or area to sensethe fingerprint (assuming the user's finger may not have been removed).

Once the additional data has been sensed, the method 400 may proceed tooperation 410. In operation 410, the display device 112 may displayfeedback corresponding to the additional data. Operation 410 may besubstantially similar to operation 404, but may vary the syntheticfingerprint or other output data based on the additional data collected.For example, one or more nodes of the synthetic fingerprint may becolored, thickened, highlighted, or the like to indicate to the userthat those additional nodes were captured by the sensor 105. Afteroperation 410, the method 400 may proceed to operation 412.

If in operation 406 no additional data is required, the method 400 mayproceed to operation 412. In operation 412, the processor 122 maydetermine whether the sensed fingerprint data matches a stored datacorresponding to a fingerprint. For example, the sensed data may becompared with a plurality of fingerprint templates. If in operation 412,the processor determines that the sensed fingerprint is a match, themethod 400 may proceed to operation 414. In operation 414, the displaydevice 112 may provide feedback to the user indicating the match of thefingerprint. For example, the synthetic fingerprint may be colored orfilled in, highlighted, or the like.

If in operation 412, the sensed fingerprint data does not correspond toa stored fingerprint, the method 400 may proceed to operation 416. Inoperation 416 the processor may determine whether additional data isrequired. For example, the processor may have determined that thefingerprint was not a match due to insufficient or inaccurate data. Inthis case, the method 400 may return to operation 408 and the sensor maycapture additional fingerprint data. However, if in operation 416, theprocessor determines additional data is not required, the method 400 mayproceed to operation 418. For example, the processor may have sufficientdata to analyze the fingerprint data and may determine that the datadoes not match any stored fingerprints. In these instances, duringoperation 418, the display device 112 may provide feedback correspondingto the mismatch or un-matched state of the fingerprint data. Aftereither operation 414 or 418, the method 400 may proceed to an end state420.

FIG. 5 is a schematic illustration of a system in accordance with anembodiment of the present invention. Described embodiments may includetouch I/O device 1001 that can receive touch input for interacting withcomputing system 1003 via wired or wireless communication channel 1002.The touch I/O device 1001 may include, for example the sensor 105 ofFIG. 1 and the display device 112 of FIG. 1 in some embodiments. In someembodiments, the touch I/O device 1001 may include the sensor 105 ofFIG. 1 , but the display device 112 may be separate. The computingsystem 1003 may include, for example the platform 110 of FIG. 1 in someembodiments. Referring again to FIG. 5 , touch I/O device 1001 may beused to provide user input to computing system 1003 in lieu of or incombination with other input devices such as a keyboard, mouse, etc. Oneor more touch I/O devices 1001 may be used for providing user input tocomputing system 1003. Touch I/O device 1001 may be an integral part ofcomputing system 1003 (e.g., touch screen on a laptop) or may beseparate from computing system 1003.

Touch I/O device 1001 may include a touch sensitive panel which iswholly or partially transparent, semitransparent, non-transparent,opaque or any combination thereof. Touch I/O device 1001 may be embodiedas a touch screen, touch pad, a touch screen functioning as a touch pad(e.g., a touch screen replacing the touchpad of a laptop), a touchscreen or touchpad combined or incorporated with any other input device(e.g., a touch screen or touchpad disposed on a keyboard) or anymulti-dimensional object having a touch sensitive surface for receivingtouch input.

In one example, touch I/O device 1001 embodied as a touch screen mayinclude a transparent and/or semitransparent touch sensitive panelpartially or wholly positioned over at least a portion of a display.According to this embodiment, touch I/O device 1001 functions to displaygraphical data transmitted from computing system 1003 (and/or anothersource) and also functions to receive user input. In other embodiments,touch I/O device 1001 may be embodied as an integrated touch screenwhere touch sensitive components/devices are integral with displaycomponents/devices. In still other embodiments a touch screen may beused as a supplemental or additional display screen for displayingsupplemental or the same graphical data as a primary display and toreceive touch input.

Touch I/O device 1001 may be configured to detect the location of one ormore touches or near touches on device 1001 based on capacitive,resistive, optical, acoustic, inductive, mechanical, chemicalmeasurements, or any phenomena that can be measured with respect to theoccurrences of the one or more touches or near touches in proximity todeice 1001. Software, hardware, firmware or any combination thereof maybe used to process the measurements of the detected touches to identifyand track one or more gestures. A gesture may correspond to stationaryor non-stationary, single or multiple, touches or near touches on touchI/O device 1001. A gesture may be performed by moving one or morefingers or other objects in a particular manner on touch I/O device 1001such as tapping, pressing, rocking, scrubbing, twisting, changingorientation, pressing with varying pressure and the like at essentiallythe same time, contiguously, or consecutively. A gesture may becharacterized by, but is not limited to a pinching, sliding, swiping,rotating, flexing, dragging, or tapping motion between or with any otherfinger or fingers. A single gesture may be performed with one or morehands, by one or more users, or any combination thereof.

Computing system 1003 may drive a display with graphical data to displaya graphical user interface (GUI). The GUI may be configured to receivetouch input via touch I/O device 1001. Embodied as a touch screen, touchI/O device 1001 may display the GUI. Alternatively, the GUI may bedisplayed on a display separate from touch I/O device 1001. The GUI mayinclude graphical elements displayed at particular locations within theinterface. Graphical elements may include but are not limited to avariety of displayed virtual input devices including virtual scrollwheels, a virtual keyboard, virtual knobs, virtual buttons, any virtualUI, and the like. A user may perform gestures at one or more particularlocations on touch I/O device 1001 which may be associated with thegraphical elements of the GUI. In other embodiments, the user mayperform gestures at one or more locations that are independent of thelocations of graphical elements of the GUI. Gestures performed on touchI/O device 1001 may directly or indirectly manipulate, control, modify,move, actuate, initiate or generally affect graphical elements such ascursors, icons, media files, lists, text, all or portions of images, orthe like within the GUI. For instance, in the case of a touch screen, auser may directly interact with a graphical element by performing agesture over the graphical element on the touch screen. Alternatively, atouch pad generally provides indirect interaction. Gestures may alsoaffect non-displayed GUI elements (e.g., causing user interfaces toappear) or may affect other actions within computing system 1003 (e.g.,affect a state or mode of a GUI, application, or operating system).Gestures may or may not be performed on touch I/O device 1001 inconjunction with a displayed cursor. For instance, in the case in whichgestures are performed on a touchpad, a cursor (or pointer) may bedisplayed on a display screen or touch screen and the cursor may becontrolled via touch input on the touchpad to interact with graphicalobjects on the display screen. In other embodiments in which gesturesare performed directly on a touch screen, a user may interact directlywith objects on the touch screen, with or without a cursor or pointerbeing displayed on the touch screen.

Feedback may be provided to the user via communication channel 1002 inresponse to or based on the touch or near touches on touch I/O device1001. Feedback may be transmitted optically, mechanically, electrically,olfactory, acoustically, or the like or any combination thereof and in avariable or non-variable manner.

Attention is now directed towards embodiments of a system architecturethat may be embodied within any portable or non-portable deviceincluding but not limited to a communication device (e.g. mobile phone,smart phone), a multi-media device (e.g., MP3 player, TV, radio), aportable or handheld computer (e.g., tablet, netbook, laptop), a desktopcomputer, an All-In-One desktop, a peripheral device, or any othersystem or device adaptable to the inclusion of system architecture 2000,including combinations of two or more of these types of devices. FIG. 6is a block diagram of one embodiment of a system arranged in accordancewith an embodiment of the present invention. The system 2000 generallyincludes one or more computer-readable media 2001, processing system2004, Input/Output (I/O) subsystem 2006, radio frequency (RF) circuitry2008 and audio circuitry 2010. These components may be coupled by one ormore communication buses or signal lines 2003. Each such bus or signalline may be denoted in the form 2003-X, where X is a unique number. Thebus or signal line may carry data of the appropriate type betweencomponents; each bus or signal line may differ from other buses/lines,but may perform generally similar operations. They system of FIG. 6 mayrepresent a more detailed example of the system 100 of FIG. 1 in someembodiments. For example, the sensor 105 of FIG. 1 may be included inthe Touch I/O device 2012 or fingerprint sensor 2042 of FIG. 6 , theplatform 110 of FIG. 1 may be implemented using the processing system2004 of FIG. 6 , and the display device may be included in the Touch I/Odevice 2012 and/or other I/O devices 2014 of FIG. 6 .

It should be apparent that the architecture shown in FIG. 6 is only oneexample architecture of system 2000, and that system 2000 could havemore or fewer components than shown, or a different configuration ofcomponents. The various components shown in FIG. 6 can be implemented inhardware, software, firmware or any combination thereof, including oneor more signal processing and/or application specific integratedcircuits.

RF circuitry 2008 is used to send and receive information over awireless link or network to one or more other devices and includeswell-known circuitry for performing this function. RF circuitry 2008 andaudio circuitry 2010 are coupled to processing system 2004 viaperipherals interface 2016. Interface 2016 includes various knowncomponents for establishing and maintaining communication betweenperipherals and processing system 2004. Audio circuitry 2010 is coupledto audio speaker 2050 and microphone 2052 and includes known circuitryfor processing voice signals received from interface 2016 to enable auser to communicate in real-time with other users. In some embodiments,audio circuitry 2010 includes a headphone jack (not shown).

Peripherals interface 2016 couples the input and output peripherals ofthe system to processor 2018 and computer-readable medium 2001. One ormore processors 2018 communicate with one or more computer-readablemedia 2001 via controller 2020. Computer-readable medium 2001 can be anydevice or medium that can store code and/or data for use by one or moreprocessors 2018. Medium 2001 can include a memory hierarchy, includingbut not limited to cache, main memory and secondary memory. The memoryhierarchy can be implemented using any combination of RAM (e.g., SRAM,DRAM, DDRAM), ROM, FLASH, magnetic and/or optical storage devices, suchas disk drives, magnetic tape, CDs (compact disks) and DVDs (digitalvideo discs). Medium 2001 may also include a transmission medium forcarrying information-bearing signals indicative of computer instructionsor data (with or without a carrier wave upon which the signals aremodulated). For example, the transmission medium may include acommunications network, including but not limited to the Internet (alsoreferred to as the World Wide Web), intranet(s), Local Area Networks(LANs), Wide Local Area Networks (WLANs), Storage Area Networks (SANs),Metropolitan Area Networks (MAN) and the like.

One or more processors 2018 run various software components stored inmedium 2001 to perform various functions for system 2000. In someembodiments, the software components include operating system 2022,communication module (or set of instructions) 2024, touch processingmodule (or set of instructions) 2026, graphics module (or set ofinstructions) 2028, one or more applications (or set of instructions)2030, which may include instructions for synthetic fingerprintgeneration, instructions for feedback regarding initial fingerprint nodeindication, and instructions for feedback regarding enrolled nodes, ashas been described above with reference to FIGS. 1-4 , and fingerprintsensing module (or set of instructions) 2038. Each of these modules andabove noted applications correspond to a set of instructions forperforming one or more functions described above and the methodsdescribed in this application (e.g., the computer-implemented methodsand other information processing methods described herein). Thesemodules (e.g., sets of instructions) need not be implemented as separatesoftware programs, procedures or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. In some embodiments, medium 2001 may store a subset of themodules and data structures identified above. Furthermore, medium 2001may store additional modules and data structures not described above.

Operating system 2022 includes various procedures, sets of instructions,software components and/or drivers for controlling and managing generalsystem tasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 2024 facilitates communication with other devicesover one or more external ports 2036 or via RF circuitry 2008 andincludes various software components for handling data received from RFcircuitry 2008 and/or external port 2036.

Graphics module 2028 includes various known software components forrendering, animating and displaying graphical objects on a displaysurface. In embodiments in which touch I/O device 2012 is a touchsensitive display (e.g., touch screen), graphics module 2028 includescomponents for rendering, displaying, and animating objects on the touchsensitive display.

One or more applications 2030 can include any applications installed onsystem 2000, including without limitation, a browser, address book,contact list, email, instant messaging, word processing, keyboardemulation, widgets, JAVA-enabled applications, encryption, digitalrights management, voice recognition, voice replication, locationdetermination capability (such as that provided by the globalpositioning system (GPS)), a music player, etc. The applications 2030may also include applications for performing functions described abovewith reference to FIGS. 1-4 , including synthetic fingerprint selectionand/or display, feedback for initial enrollment node, and feedback forindicating captured nodes.

Touch processing module 2026 includes various software components forperforming various tasks associated with touch I/O device 2012 includingbut not limited to receiving and processing touch input received fromI/O device 2012 via touch I/O device controller 2032.

System 2000 may further include fingerprint sensing module 2038 forperforming the method/functions as described herein in connection withFIGS. 1-4 . Fingerprint sensing module 2038 may at least function toperform various tasks associated with the fingerprint sensor, such asreceiving and processing fingerprint sensor input. The fingerprintsensing module 2038 may also control certain operational aspects of thefingerprint sensor 2042, such as its capture of fingerprint data and/ortransmission of the same to the processor 2018 and/or secure processor2040. Module 2038 may also interact with the touch I/O device 2012,graphics module 2028 or other graphical display. Module 2038 may beembodied as hardware, software, firmware, or any combination thereof.Although module 2038 is shown to reside within medium 2001, all orportions of module 2038 may be embodied within other components withinsystem 2000 or may be wholly embodied as a separate component withinsystem 2000.

I/O subsystem 2006 is coupled to touch I/O device 2012 and one or moreother I/O devices 2014 for controlling or performing various functions.Touch I/O device 2012 communicates with processing system 2004 via touchI/O device controller 2032, which includes various components forprocessing user touch input (e.g., scanning hardware). One or more otherinput controllers 2034 receives/sends electrical signals from/to otherI/O devices 2014. Other I/O devices 2014 may include physical buttons,dials, slider switches, sticks, keyboards, touch pads, additionaldisplay screens, or any combination thereof.

If embodied as a touch screen, touch I/O device 2012 displays visualoutput to the user in a GUI. The visual output may include text,graphics, video, and any combination thereof. Some or all of the visualoutput may correspond to user-interface objects. Touch I/O device 2012forms a touch-sensitive surface that accepts touch input from the user.Touch I/O device 2012 and touch screen controller 2032 (along with anyassociated modules and/or sets of instructions in medium 2001) detectsand tracks touches or near touches (and any movement or release of thetouch) on touch I/O device 2012 and converts the detected touch inputinto interaction with graphical objects, such as one or moreuser-interface objects. In the case in which device 2012 is embodied asa touch screen, the user can directly interact with graphical objectsthat are displayed on the touch screen. Alternatively, in the case inwhich device 2012 is embodied as a touch device other than a touchscreen (e.g., a touch pad); the user may indirectly interact withgraphical objects that are displayed on a separate display screenembodied as I/O device 2014.

Touch I/O device 2012 may be analogous to the multi-touch sensitivesurface described in the following U.S. Pat. No. 6,323,846 (Westerman etal.), U.S. Pat. No. 6,570,557 (Westerman et al.), and/or U.S. Pat. No.6,677,932 (Westerman), and/or U.S. Patent Publication 2002/0015024A1,each of which is hereby incorporated by reference in its entirety forany purpose.

Embodiments in which touch I/O device 2012 is a touch screen, the touchscreen may use LCD (liquid crystal display) technology, LPD (lightemitting polymer display) technology, OLED (organic LED), or OEL(organic electro luminescence), although other display technologies maybe used in other embodiments.

Feedback may be provided by touch I/O device 2012 based on the user'stouch input as well as a state or states of what is being displayedand/or of the computing system. Feedback may be transmitted optically(e.g., light signal or displayed image), mechanically (e.g., hapticfeedback, touch feedback, force feedback, or the like), electrically(e.g., electrical stimulation), olfactory, acoustically (e.g., beep orthe like), or the like or any combination thereof and in a variable ornon-variable manner.

System 2000 also includes power system 2045 for powering the varioushardware components and may include a power management system, one ormore power sources, a recharging system, a power failure detectioncircuit, a power converter or inverter, a power status indicator and anyother components typically associated with the generation, managementand distribution of power in portable devices.

In some embodiments, peripherals interface 2016, one or more processors2018, and memory controller 2020 may be implemented on a single chip,such as processing system 2004. In some other embodiments, they may beimplemented on separate chips.

In addition to the foregoing, the system 2000 may include a secureprocessor 2040 in communication with a fingerprint sensor 2042, via afingerprint I/O controller 2044. The operation of these various elementswill now be described.

The fingerprint sensor 2042 may operate to capacitively capture a seriesof images, or nodes. When taken together, these nodes may form afingerprint. The full set of nodes may be referred to herein as a“mesh.”

Each node in the mesh may be separately captured by the fingerprintsensor 2042, which may be an array sensor. Generally, there is someoverlap between images in nodes representing adjacent portions of afingerprint. Such overlap may assist in assembling the fingerprint fromthe nodes, as various image recognition techniques may be employed touse the overlap to properly identify and/or align adjacent nodes in themesh.

Sensed fingerprint data may be transmitted through the fingerprint I/Ocontroller 2044 to the processor 2018 and/or the secure processor 2040.In some embodiments, the data is relayed from the fingerprint I/Ocontroller 2044 to the secure processor 2040 directly. Generally, thefingerprint data is encrypted by any of the fingerprint sensor 2042, thefingerprint I/O controller 2044 or another element prior to beingtransmitted to either processor. The secure processor 2040 may decryptthe data to reconstruct the node.

Fingerprint data, either as nodes or meshes, may be stored in thecomputer-readable medium 2001 and accessed as necessary. In someembodiments, only the secure processor 2040 may access storedfingerprint data, while in other embodiments either the secure processoror the processor 2018 may access such data.

FIG. 7 illustrates an example method of fingerprint sensing, asdescribed herein. The method begins with receiving fingerprint data fromat least one sensor, step 700. Example approaches that may be used forreceiving the fingerprint data are described in detail above. Theseexample approaches may include utilizing at least one applicationprocessor.

The received fingerprint data is provided to at least one processor,step 702. The processor(s), as described in detail above, may include atleast one secure enclave processor. The processor(s) decodes thefingerprint data, step 704, and provides a signal indicative of at leastone matched node, step 706. Various example processes whereby theprocessor(s) performs these operations are described in detail above.

Responsive to receipt of the signal indicative of the matched node(s),at least a portion of a synthetic fingerprint image is presented via atleast one display device, step 708. The portion of the syntheticfingerprint image presented corresponds to the matched node(s), as maybe determined utilizing at least one application processor(s). Examplesof the operation of application processors performing this step aredescribed above.

Although embodiments have been described herein with respect toparticular configurations and sequences of operations, it should beunderstood that alternative embodiments may add, omit, or changeelements, operations and the like. Accordingly, the embodimentsdisclosed herein are meant to be examples and not limitations.

We claim:
 1. A method, comprising: at an electronic device with adisplay and a fingerprint sensor: capturing first fingerprintinformation using the fingerprint sensor; in response to capturing thefirst fingerprint information, displaying, on the display, a fingerprintrepresentation with a first portion of the fingerprint representationdisplayed differently from remaining portions of the fingerprintrepresentation, the remaining portions of the fingerprint representationincluding a second portion of the fingerprint representation; afterdisplaying the fingerprint representation with the first portion of thefingerprint representation displayed differently from remaining portionsof the fingerprint representation, capturing second fingerprintinformation using the fingerprint sensor; and in response to capturingthe second fingerprint information, changing an appearance of the secondportion of the fingerprint representation.
 2. The method of claim 1,wherein the second portion of the fingerprint representation is changedto be displayed differently from remaining portions of the fingerprintrepresentation other than the first portion of the fingerprintrepresentation and the second portion of the fingerprint representation.3. The method of claim 1, wherein the first portion of the fingerprintrepresentation and the second portion of the fingerprint representationare changed in the same manner.
 4. The method of claim 1, wherein thefirst portion of the fingerprint representation and the second portionof the fingerprint representation are changed by changing a property ofone or more lines in the fingerprint representation.
 5. The method ofclaim 1, wherein the fingerprint representation is a graphical elementthat includes elements that represent fingerprint ridges.
 6. The methodof claim 5, wherein the first portion of the fingerprint representationand the second portion of the fingerprint representation are changed bychanging a property of one or more elements that represent fingerprintridges.
 7. The method of claim 6, wherein the property is a linethickness.
 8. An electronic device comprising: a fingerprint sensor; anda display; wherein the electronic device is configured to: capture firstfingerprint information using the fingerprint sensor; in response tocapturing the first fingerprint information, display, on the display, afingerprint representation with a first portion of the fingerprintrepresentation displayed differently from remaining portions of thefingerprint representation, the remaining portions of the fingerprintrepresentation including a second portion of the fingerprintrepresentation; after displaying the fingerprint representation with thefirst portion of the fingerprint representation displayed differentlyfrom remaining portions of the fingerprint representation, capturesecond fingerprint information using the fingerprint sensor; and inresponse to capturing the second fingerprint information, change anappearance of the second portion of the fingerprint representation. 9.The electronic device of claim 8, wherein the second portion of thefingerprint representation is changed to be displayed differently fromremaining portions of the fingerprint representation other than thefirst portion of the fingerprint representation and the second portionof the fingerprint representation.
 10. The electronic device of claim 8,wherein the first portion of the fingerprint representation and thesecond portion of the fingerprint representation are changed in the samemanner.
 11. The electronic device of claim 8, wherein the first portionof the fingerprint representation and the second portion of thefingerprint representation are changed by changing a property of one ormore lines in the fingerprint representation.
 12. The electronic deviceof claim 8, wherein the fingerprint representation is a graphicalelement that includes elements that represent fingerprint ridges. 13.The electronic device of claim 12, wherein the first portion of thefingerprint representation and the second portion of the fingerprintrepresentation are changed by changing a property of one or moreelements that represent fingerprint ridges.
 14. The electronic device ofclaim 13, wherein the property is a color.
 15. A non-transitory computerreadable medium storing instructions that when executed by an electronicdevice cause the electronic device to: capture first fingerprintinformation using a fingerprint sensor; in response to capturing thefirst fingerprint information, display, on a display, a fingerprintrepresentation with a first portion of the fingerprint representationdisplayed differently from remaining portions of the fingerprintrepresentation, the remaining portions of the fingerprint representationincluding a second portion of the fingerprint representation; afterdisplaying the fingerprint representation with the first portion of thefingerprint representation displayed differently from remaining portionsof the fingerprint representation, capture second fingerprintinformation using the fingerprint sensor; and in response to capturingthe second fingerprint information, change an appearance of the secondportion of the fingerprint representation.
 16. The non-transitorycomputer readable medium of claim 15, wherein the second portion of thefingerprint representation is changed to be displayed differently fromremaining portions of the fingerprint representation other than thefirst portion of the fingerprint representation and the second portionof the fingerprint representation.
 17. The non-transitory computerreadable medium of claim 15, wherein the first portion of thefingerprint representation and the second portion of the fingerprintrepresentation are changed in the same manner.
 18. The non-transitorycomputer readable medium of claim 15, wherein the first portion of thefingerprint representation and the second portion of the fingerprintrepresentation are changed by changing a property of one or more linesin the fingerprint representation.
 19. The non-transitory computerreadable medium of claim 15, wherein the fingerprint representation is agraphical element that includes elements that represent fingerprintridges.
 20. The non-transitory computer readable medium of claim 19,wherein the first portion of the fingerprint representation and thesecond portion of the fingerprint representation are changed by changingat least one of a dash pattern or a shading of one or more elements thatrepresent fingerprint ridges.